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Statement of Inquiry 
1. Ihave been asked to evaluate the report of Dr. William Briggs, dated November 23, 
2020. 


2. Iam compensated at the rate of $550 an hour. 


Summary Assessment 


3. Based on my review, I find the estimates and analyses in Dr. Briggs report to be 
unreliable and the analysis is not up to scientific standards of survey research, data analysis, or 
election analysis. There are substantial errors in the design of the survey and errors and 
inconsistencies in the data used in the analysis that are of sufficient magnitude to invalidate any 
calculations or estimates based on these data. The survey design and implementation fail to meet 
basic scientific standards of survey research and statistical analysis of surveys. And, the 
interpretation of the data does not account for obvious and important features of absentee voting, 
including permanent absentee voters who do not need to request ballots to receive them and late, 
rejected, invalid, and spoiled absentee ballots. The errors in design, analysis, and interpretation 
of the data are so massive that there is no foundation for drawing any conclusions or inferences 


based on Dr. Briggs’ report. 


4. In his report, Dr. Briggs evaluates survey data that was provided to him by a third 
party and assumes that “the respondents [to the survey] are representative and the data are 
accurate.” There is no indication in his report that any analysis was conducted by him or by 


those who provided the data to him to verify the correctness or integrity of the data he was 
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provided, the quality of the survey, or the representativeness of the sample on which he based his 
analysis. It is standard scientific practice in the field of survey research to give careful scrutiny 
to data before conducting any statistical analyses, including understanding the structure and 
wording of the survey questions, the sampling method and response rate, and the characteristics 


of the sample, such as demographic and behavioral indicators. 


5. In his report, Dr. Briggs defines two types of errors. Those people who received 
absentee ballots even though the survey indicates that they did not request an absentee ballot are 
called Error #1. Those people who returned absentee ballots even though the election office did 
not record an absentee vote from them are called Error #2. These two errors combined he calls 
“troublesome ballots.” Based on the information in Dr. Briggs’ report, it is my conclusion, that 
neither assumption is justified. The estimates of Error #1 and Error #2 that he presents reflect 
defects in the design of the survey, fatal data errors evident in the survey toplines, calculation 
errors, and errors in the interpretation of the data. It is my professional judgment that none of the 


estimates and projections in his report are valid. 


6. The design of the survey contaminates the data and any estimates, rendering them 
invalid. Specifically, in Question | of the survey the surveyor asks to speak to a specific person. 
Some of the respondents are flagged as “Reached Target,” while others are flagged as 
“Uncertain” or “What is this about?” Both groups of people (Reached Target and Uncertain) are 
then asked Question 2, Did you request an absentee ballot? This is a serious survey design error, 
because some or perhaps all of the people flagged as “Uncertain” are not the Target of the 


interview. As a result, the design of the very first question of the survey allows some people 
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who were not the Target to be treated as if they were the Target in the remaining questions. This 
leads to the contamination of all results. This fact is evident in the Topline Tables that were 


attached to Dr. Briggs’ report. 


7. The survey suffers from ambiguously worded questions, which introduces 
measurement errors in any estimates. Question 2 asks respondents whether they requested an 
absentee ballot. They question does not clearly state what a request for an absentee ballot means. 
Perhaps the most troublesome example for this question is how those who are permanent 
absentee voters ought to respond to it. Permanent absentee voters are sent an absentee ballot 
automatically; they do not need to request a ballot for a particular election in order for the 
election office to mail one to them. Michigan allows for permanent absentee voting, as do three 
other states in his analysis -- Arizona, Pennsylvania, and Wisconsin. Georgia allows permanent 
absentee voting each election cycle for those 65 years of age or older. Both “yes” and “no” may 
be viewed as correct answers to this question. Respondents who are permanent absentee voter 
might respond yes because they did sign up for that status, or they might as correctly respond no 
because they did not have to request a ballot in order to have one sent to them. The 
questionnaire provides no way to clarify such cases. There is no follow up question to 
disambiguate permanent absentee voters from others. This is just one example of the substantial 


problems with the wording and structure of Question 2. 


7. The wording of Question 3 is also very problematic. First, it does not ascertain 
whether the ballot was mailed back in a timely manner so as to be included in the record of 


ballots cast. Some or possibly all of the cases in question are late ballots, and thus not 
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necessarily included in the absentee vote record. Second, Question 3 asks whether someone 
voted. Survey questions asking whether someone voted are notoriously subject to social 


desirability biases that lead to inflation in the estimated number of voters. 


8. There are errors and inconsistencies in survey data for every state, including for the 
state of Michigan. Appended to Dr. Briggs’ report are a series of tables, called Topline Tables, 
for each state. The Topline Tables, or toplines for short, provide the basic statistics about the 
survey reported for each question, as well as the questions themselves and the response 
categories for each question. There are errors in the spreadsheet of toplines indicating data 
inconsistencies, including for the State of Michigan. For example, in the topline tables for 
Michigan, the number of respondents to Question 1 who are supposed to be asked Question 2 
does not sum to the number of respondents to Question 2. The same accounting discrepancy 
arises in Arizona, Pennsylvania, and Wisconsin, but not in Georgia. In Michigan and Arizona, 
there are too many respondents to Question 2. In Pennsylvania and Wisconsin, there are too few 
respondents to Question 2. These errors infect and bias responses to Q2 and Q3. Generally, 
such errors indicate fundamental problems with the management of the survey and the databases 
generated by the survey. In standard survey practice, the presence of discrepancies in these 
topline tables indicates fatal flaws in the data, and lead researchers to attempt to clarify the 
problems and possibly discard the data altogether. Dr. Briggs’ report makes no mention of these 
inconsistencies and errors and assumes that the underlying data are correct. These errors and 


inconsistencies reveal that the data are not correct. 
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9. The survey has extremely low response rates. The response rate for the Michigan 
survey is .008, just eight tenths of one percent. That means that 99.2 percent of people who the 
survey firm sought to interview in the State of Michigan could not be contacted, refused to 
participate, or were not in fact the correct person. Surveys with such a low response rate are not 
accepted in scientific publications, except on rare occasions and with proper analyses that ensure 
that the respondents in fact are representative. When researchers have low response rates, they 
must offer affirmative proof of representativeness or attempt to correct for biases. Neither is 


done here. 


10. In performing his analysis, Dr. Briggs extrapolates from a poorly designed survey 
with an extraordinarily high non-response rate and evident data errors and inconsistencies. The 
high non-response rate, the data errors, and the survey design flaws are all evident in the topline 
tables that Dr. Briggs appended to his report. These data should not have been relied on for this 
analysis given that they are not correct, and the respondents to the survey are highly unlikely to 


represent the population in question. This is not up to scientific standards. 


11. The interpretation of the data as evidence of “errors” and “troublesome ballots” fails 
to account for the rules and realities of absentee voting. First, Dr. Briggs calls Error #1 absentee 
ballots that were received by voters but were not “requested.” This interpretation fails to consider 
permanent absentee voters, who receive ballots without requesting them. All five states in his 
report allow for permanent absentee voting for some or all registrants. Second, Dr. Briggs calls 
Error #2 ballots that were sent by voters but not recorded. This interpretation fails to account for 


late, undeliverable, rejected, and spoiled ballots. Most jurisdictions, for example, do not record 
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late ballots in the tally of returned absentee ballots. The results in his analysis, if they are real, 
are likely the consequence of the normal practice of absentee voting in compliance with state 


election procedures and laws. 


II. Qualifications 


12. I am the Frank G. Thompson Professor of Government in the Department of 
Government at Harvard University in Cambridge, MA. Formerly, I was an Assistant Professor at 
the University of California, Los Angeles, and I was Professor of Political Science at the 
Massachusetts Institute of Technology, where I held the Elting R. Morison Chair and served as 
Associate Head of the Department of Political Science. I am the Principal Investigator of the 
Cooperative Congressional Election Study (CCES), a survey research consortium of over 250 
faculty and student researchers at more than 50 universities, directed the Caltech/MIT Voting 
Technology Project from its inception in 2000 through 2004, and served on the Board of Overseers 
of the American National Election Study from 1999 to 2013. I am an election analyst for and 
consultant to CBS News’ Election Night Decision Desk. Iam a member of the American Academy 
of Arts and Sciences (inducted in 2007). My curriculum vitae is attached to this report as Appendix 


A. 


13. Ihave worked as a consultant to the Brennan Center in the case of McConnell v. FEC, 
540 U.S. 93 (2003). I have testified before the U.S. Senate Committee on Rules, the U.S. Senate 
Committee on Commerce, the U.S. House Committee on Science, Space, and Technology, the 


U.S. House Committee on House Administration, and the Congressional Black Caucus on matters 
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of election administration in the United States. I filed an amicus brief with Professors Nathaniel 
Persily and Charles Stewart on behalf of neither party to the U.S. Supreme Court in the case of 
Northwest Austin Municipal Utility District Number One v. Holder, 557 U.S. 193 (2009) and an 
amicus brief with Professor Nathaniel Persily and others in the case of Evenwel v. Abbott 138 S.Ct. 
1120 (2015). Ihave served as a testifying expert for the Gonzales intervenors in State of Texas v. 
United States before the U.S. District Court in the District of Columbia (No. 1:11-cv-01303); the 
Rodriguez plaintiffs in Perez v. Perry, before the U.S. District Court in the Western District of 
Texas (No. 5:11-cv-00360); for the San Antonio Water District intervenor in LULAC v. Edwards 
Aquifer Authority in the U.S. District Court for the Western District of Texas, San Antonio 
Division (No. 5:12cv620-OLG); for the Department of Justice in State of Texas v. Holder, before 
the U.S. District Court in the District of Columbia (No. 1:12-cv-00128); for the Guy plaintiffs in 
Guy v. Miller in U.S. District Court for Nevada (No. 11-OC-00042-1B); for the Florida 
Democratic Party in In re Senate Joint Resolution of Legislative Apportionment in the Florida 
Supreme Court (Nos. 2012-CA-412, 2012-CA-490); for the Romo plaintiffs in Romo v. Detzner 
in the Circuit Court of the Second Judicial Circuit in Florida (No. 2012 CA 412); for the 
Department of Justice in Veasey v. Perry, before the U.S. District Court for the Southern District 
of Texas, Corpus Christi Division (No. 2:13cv00193); for the Harris plaintiffs in Harris v. 
McCrory in the U. S. District Court for the Middle District of North Carolina (No. 
1:2013cv00949); for the Bethune-Hill plaintiffs in Bethune-Hill v. Virginia State Board of 
Elections in the U.S. District Court for the Eastern District of Virginia (No. 3: 2014cv00852); for 
the Fish plaintiffs in Fish v. Kobach in the U.S. District Court for the District of Kansas ( No. 2:16- 
cv-02105-JAR); and for intervenors in Voto Latino, et al. v. Hobbs, in the U.S. District Court for 


the District of Arizona (No. 2:19-cv-05685-DWL). I served as an expert witness and filed an 


Case 2:20-cv-13134-LVP-RSW ECF No. 36-1, PagelD.2529 Filed 12/02/20 Page 10 of 60 


Affidavit in the North Carolina State Board of Elections hearings regarding absentee ballot fraud 
in the 2018 election for Congressional District 9 in North Carolina. I currently am an expert 


witness Wood v. Raffensperger, in Fulton County, Georgia, Superior Court, (No. 2020CV342959). 


14. My areas of expertise include American government, with particular expertise in 
electoral politics, representation, and public opinion, as well as statistical methods in social 
sciences and survey research methods. I have authored numerous scholarly works on voting 
behavior and elections, the application of statistical methods in social sciences, legislative politics 
and representation, and distributive politics. This scholarship includes articles in such academic 
journals as the Journal of the Royal Statistical Society, American Political Science Review, 
American Economic Review, the American Journal of Political Science, Legislative Studies 
Quarterly, Quarterly Journal of Political Science, Electoral Studies, and Political Analysis. I have 
published articles on issues of election law in the Harvard Law Review, Texas Law Review, 
Columbia Law Review, New York University Annual Survey of Law, and Election Law Journal, 
for which I am a member of the editorial board. I have coauthored three scholarly books on 
electoral politics in the United States, The End of Inequality: Baker v. Carr and the Transformation 
of American Politics, Going Negative: How Political Advertising Shrinks and Polarizes the 


Electorate, and The Media Game: American Politics in the Media Age. I am coauthor with 


Benjamin Ginsberg, and Ken Shepsle of American Government: Power and Purpose. 


II. Sources 
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15. Ihave relied on the report of Dr. William Briggs, especially the appended Topline 


Tables. 


16. [have relied on the Election Assistance Commission, “Election Administration and 
Voting Survey (EAVS) for 2018: https://www.eac.gov/research-and-data/studies-and-reports. I 
present data from 2018 because it is the most recent federal election for which data on absentee 
and permanent absentee voting is available. The 2018 data are instructive about the magnitude 
of permanent absentee voters and of the magnitude of unreturned, late, rejected, and spoiled 


absentee ballots. The 2020 data are not yet reported. 


17. [have relied on the report of Matthew Braynard in King v. Whitmer, No. 2:20-cv- 


13134-LVP-RSW (ED. Mich.2020). 


IV. Findings 


18. In my professional judgment there are fundamental flaws in the survey design, and 
survey data set out in Dr. Briggs’ report. These flaws created biases in the estimates and 
analyses that are sufficiently large to completely explain the data that Dr. Briggs presents as 
nothing more than errors in the data collection process. Perhaps most troubling the survey is 
likely highly unrepresentative because it has a response rate less than 1 percent; the survey data 
are contaminated by respondents who should not have been included in the survey, and the basic 
data in the Topline summaries of the data do not add up, indicating fatal flaws in the 


implementation of the survey. 
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A. Critique of Interpretation 


i. The survey data and its interpretation does not account for Permanent Absentee 


and Early Voters (PEV). 


19. The analysis of Question 2 is used to estimate the number of people who received but 


did not request an absentee ballot. Dr. Briggs calls this Error #1. 


20. The interpretation of these data as an Error in balloting does not account for the 
presence of a large number of Permanent Absentee and Early Voters (PEVs) in Arizona, 
Michigan, Pennsylvania, and Wisconsin, and “rollover” absentee voters in Georgia. PEVs are 
automatically sent their absentee ballots. They do not need to request that a ballot be sent for a 
particular election. Dr. Briggs’ interpretation of the responses to Question 2 as indicative of an 
Error #1 fails to take into account the large number of people in Michigan and in the other four 


states in his study who are PEVs. There are other problems with Question 2 as discussed below. 


21. There are a sizable number of PEVs in all of the states under study. Table | presents 
data from the number of absentee ballots sent in 2018 and the number of permanent absentee 
ballots sent in to voters in Arizona, Georgia (rollover ballots), Michigan, Pennsylvania, and 
Wisconsin. According to figures from the Election Assistance Commission for 2018, there were 


over half a million PEVs in the state of Michigan in 2018. The number of permanent absentee 


10 
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ballots sent in Arizona, Michigan, and Wisconsin far exceeds the estimated Error #1 in the first 
table in Dr. Briggs’ report. Those data cover 2018 and are presented to indicate the likely 


magnitude of PEVs in the states in 2020. 


22. The number of PEVs in Michigan was even larger in 2020 Preliminary figures (as of 
September, 2020) from the Office of the Secretary of State of Michigan reveal that the number of 
PEVs in the November 2020 election was approximately 2 million, and that PEVs accounted for 
roughly two-thirds of absentee ballots sent this for the 2020 general election in the State of 
Michigan.' The other states saw similarly large numbers of PEVs in 2020. For example, the 
State of Georgia, which allows voters who are disabled or over 65 to sign up to have absentee 


ballots sent to them, reported at least 582,000 “rollover” ballots in Georgia in 2020.7 


23. The number of registered voters who were sent a ballot without having to request one 
in the State of Michigan far exceeds the total number of absentee ballots that Dr. Briggs 
classifies as Error #1. The number of PEVs was 2 million in the November 2020 election. The 
maximum number of people who received an absentee ballot but said that they did not request 


one in Dr. Briggs’ assessment is 36,529. 


1 Dave Boucher, “Michigan sets record for number of absentee ballot requests for Nov. election” Detroit Free Press 
September 10, 2002. https://www.freep.com/story/news/politics/elections/2020/09/10/michigan-absentee- 
voting-election-november-2020/5768768002/ 

Ray Hole, “Michigan receives record number of requests for absentee ballots,” wwmt.com, September 10, 2020, 
https://wwmt.com/news/local/michigan-receives-record-number-of-request-for-absentee-ballots 

2 Stephen Fowler, “Nearly 800,000 Georgians Have Already Requested Absentee Ballots for November” GA Today 
gpb.org, September 2, 2020. https://www.gpb.org/news/2020/09/02/nearly-800000-georgians-have-already- 
requested-absentee-ballots-for-november 
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24. The survey makes no effort to distinguish PEVs from other sorts of absentee voters. 


Not accounting for PEVs is a serious error in survey design and interpretation of the survey 


numbers. 






































Table 1. Permanent Absentee Voters in Arizona, Georgia, Michigan, Pennsylvania, and 
Wisconsin in 2018 
Total Absentee Permanent Absentee | Permanent Absentee 
Ballots Sent Ballots Sent Ballots as a Percent 
(i.e., ballots sent of Total 
automatically without 
a specific ballot 
request) 
Arizona 2,672,384 2,545,198 95.2% 
Georgia 281,490 ia * 
Michigan 1,123,415 549,894 48.9% 
Pennsylvania 216,575 6,340 2.9% 
Wisconsin 168,788 54,113 32.1% 
Source: EAC, EAVS 2018. 
Note: * means no data reported. 








ii. The interpretation of Question 3 fails to account for the proper handling of 


late, invalid, and spoiled absentee ballots by Local Election Offices. 


12 
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25. The analysis of Question 3 is used to estimate the number of people who stated that 
they returned an absentee ballot but for whom no vote was recorded. Dr. Briggs calls this Error 


#2. 


26. The interpretation does not account absentee ballots that are in fact not received or 
counted by election officers because the ballots are not returned by the postal system, are 
spoiled, are returned late, or are rejected. Such ballots are the obvious explanation for the data 
observed, but no effort in the survey or the analysis is made to ascertain the likelihood that these 


ballots. There are further problems with Question 3, as discussed below. 


27. It is my experience researching elections over the past two decades that “uncounted” 
absentee ballots are a normal part of the election process. Table 2 presents counts of rejected, 
late, undelivered, and voided absentee ballots in Arizona, Georgia, Michigan, Pennsylvania, and 
Wisconsin for 2018, the most recent federal election for which systematic data on absentee 
voting are available. An undeliverable absentee ballot is one that was returned to the election 
office as not being deliverable to the address on the voter registration lists. The final column 
presents the number of sent absentee ballots for which the status of a ballot sent by the election 
office to a voter was not received and its status is not known. These are likely ballots that simply 
were not returned by voters or were lost or delayed in the US Postal System, as happens in every 


election in my experience. 
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28. The magnitude of ballots that are returned to the office but are rejected, spoiled or late 


is quite large. The sum of these columns is comparable in magnitude to the magnitude of “Error 


#2” in Dr. Briggs’ report. These figures are not definitive of the numbers in 2020, which have 


not yet been reported. Rather they are demonstrative of the fact that there are sound, 


documented administrative reasons that returned absentee ballots are not recorded as having 


voted, especially tardiness, spoilage, and rejection for lack of signatures, valid envelopes, and the 


like. These are ballots that are not allowed to be counted under law, and they are comparable in 


magnitude to the estimates of the Error #2 reported by Dr. Briggs for each state. 





Pennsylvania, and Wisconsin in 2018 


Table 2. Rejected, Undelivered, Voided, and Late Absentees in Arizona, Georgia, Michigan, 



































Rejected Undeliverable | Spoiled/Voided Late Status 
Absentee Absentee Absentee Absentee | Unknown 
Ballots Ballots Ballots Ballots 
Arizona 8,567 102,896 27,804 2,915 642,210 
Georgia 7,512 2,322 252 3,525 36,255 
Michigan 6,013 791 19,679 2,207 41,120 
Pennsylvania 8,714 * * 8,162 20,622 
Wisconsin 2,517 1,718 2,794 1,445 12,407 








Source: EAC, EAVS 2018. 


Note: * means no data reported. 
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B. Critique of Survey Design 


29. Dr. Briggs offers no assessment of the design of the survey that generated the data 
that he presents. Rather, he assumes that the data are accurate. Also, there is no report of the 
survey design, beyond the information embedded in the topline tables. It would be standard for 
any scientifically sound report of survey data to describe fully the survey instrument used in a 


study and to make it publicly available. 


30. It is my understanding that Matthew Braynard designed and conducted these surveys. 
The methodology that he used is described in an expert by Mr. Braynard in King v. Whitmer, No. 
2:20-cv-13134-LVP-RSW (ED. Mich.2020). I evaluated that report as well as Dr. Briggs’ 


report. 


i. The surveys have unacceptably high non-response rates. 


31. The response rate to the survey is measured as the number of people who answered 
the first substantive question (Q2) in the survey divided by the number of people who the 
surveyor sought to contact. The response rate for the Michigan survey is .008, meaning 99.2 
percent of people did not respond to this survey. The response rate is similarly low in the other 


four states. It is .006 in Arizona, .004 in Georgia, .015 in Pennsylvania, and .004 in Wisconsin. 
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These are extremely low response rates, and such low response rates pose a critical threat to any 


inferences one might draw from the data. 


32. Mr. Braynard, in his report, identifies that the survey attempts to interview all 
registered voters who were recorded as requesting but not returning an absentee ballot. Mr. 
Braynard’s firm attempted to match phone numbers to records of registered voters in each of the 
states and then attempted to interview all of the people associated with each registration record 


of interest. 


33. The appendix to Dr. Briggs’ report presents Unreturned_Absentee Live ID Topline 
tables for each of the five states being studied, including Michigan. It is evident from the 
Topline tables that a there are significant shortfalls in the ability of the survey firm to match 
phone numbers to registration records. The Data Loads correspond to the number of matched 
phone numbers that were loaded into the survey system to be called. They are only a fraction of 


the population of all Unreturned Absentees. 


34. The toplines also list Completes. These are phone numbers for which an interview 
commenced, an answering machine was reached, or a returned call was requested. For the State 
of Michigan, there were 70,030 Data Loads for Michigan, which is just half of the 139,190 
unreturned absentee ballots in the State. Of the Data Loads, 1.e., attempted calls, there were only 
3,815 people who either responded to the survey, were left a message on an answering machine, 
or refused. These are called “Completes,” even though most refused the call. Only 5 percent of 


the people that Mr. Braynard attempted to interview even got to the point of being classified as a 
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Complete. That is, 95 percent of the attempted survey contacts had already failed before the 


survey had begun. 


35. There is no description in Dr. Briggs’ report of the generation of “Data Loads” or the 
methodology for determining matches of phone numbers to registration records. No explanation 
is offered as to why the researchers could load only 70,030 phone numbers, rather than attempt 
calls to all 139,190 Michigan registrants who did not return their absentee ballots. Based on my 
experience conducting survey research, the likely problem is the difficult matching phone 
numbers to voter files and the large number voter file records that list no numbers. Mismatches, 
either false positives or false negatives, will generate errors in surveys. Incorrectly matched 
phone numbers will lead the survey to interview the wrong person (a false positive), and errors in 
matching may lead to researcher to exclude the person from the survey when in fact a valid 
number could have been found (a false negative).? Errors in phone matching make it entirely 
possible that the survey interviews the wrong person. Past research on phone surveys using 
registration-based samples, such as those reported by Dr. Briggs, find very high rates with which 
the wrong person answers a call, ranging from 30 to 60 percent.* And, as discussed below, there 


is evidence of such incorrect interviews in these studies. 


36. Dr. Briggs offers no analysis of why the survey failed to identify a higher number of 


valid phone numbers for the people the researchers sought to interview. There is no analysis 


3 Alan S. Gerber and Donald P. Green, “Can Registration-Based Sampling Improve the Accuracy of Midterm Election 
Forecasts?” Public Opinion Quarterly 70 (2006): 197-223, esp. page 202. 

4 Pew Research Center, “Comparing Survey Sampling Strategies: Random-Digit Dialing vs. Voter Files,” 2018. 
https://www.pewresearch.org/methods/2018/10/09/comparing-survey-sampling-strategies-random-digit-dial-vs- 


voter-files/, See page 25-26. 
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comparing the characteristics of those people for whom a valid phone number could be found 


and those people for whom a valid phone number could not be found. 


37. Once the survey commences, there is first a screener question to determine whether 
the person interviewed should continue with the interview. That is Question 1. Question 2 asks 
“Did you request an Absentee Ballot in the State of <state name>?” People could answer Yes, 
No, some other answer, Refuse to answer, or Hang up. This is the first substantive question for 


the purpose of Dr. Briggs’ analysis, as it is used to measure Error #1. 


38. The response rate to the survey is the number of valid responses to Question 2. That 
is the total number of responses to the question, less the number of people who refused or hung 
up. The second column of Table 3 is the percent of people the researchers sought to interview 


(all Unreturned Absentee Ballots) who ultimately gave a valid response to Question 2. 


39. The response rate for Michigan is just .008, or eight-tenths of one percent. 


40. Once the entire survey process had been completed over 99 percent of people who the 


researcher sought to interview were not interviewed in Michigan. That was true in Arizona, 


Georgia, and Wisconsin. In Pennsylvania, 98.5 percent of those that the researchers set out to 


study were ultimately not included in the study for one reason or another. 


41. In most disciplines of study that I am familiar with these surveys would not a 


scientifically acceptable or reliable samples simply because of their exceedingly low response 
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rates. For example, I am as associate editor of the Harvard Data Sciences Review, which 
broadly covers fields of statistics and data sciences, and specialty fields, such as, political 
science, public opinion, survey methodology, and economics, in which I have published. Papers 


with such high non-responses are rejected on their face as not plausibly valid studies. 


42. Dr. Briggs’ assumption that those who responded to the question are representative of 
the relevant population under study (i.e., the other 99 percent of people who could not or would 
not participate in the survey) is not warranted. When survey have high non-response rates, it is 
standard practice to analyze information about the sample and the target population, such as 
demographic characteristics or behavioral and attitudinal statistics, to confirm that the 
assumption of representativeness of a sample can be maintained. That is true even when the 
response rates are quite high. When the response rates are very low, such an analysis is a 
necessity in order to determine whether there is any scientific value to the survey. No such 


analysis is offered here. 





Table 3. Response Rates to Surveys Reported by Dr. William Briggs 




















State “Completes”’/ Question 2 Valid Response/ 
Unreturned Absentee Ballots | Unreturned Absentee Ballots 

Arizona O11 .006 

Georgia 110 004 

Michigan 027 008 

Pennsylvania .109 O15 

Wisconsin 048 004 
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Note: Ballots is the number of registered voters the survey sought to reach. See Table | of 
Briggs’ report. 

“Completes” is the number of “complete” contacts in the firsts part of each state’s topline 
report. 

Question 2 Response is the number of respondents who answered Question 2 and did not 


Refuse or Hangup. 











il. The survey has an unacceptably high interview breakoff rate. 


43. The breakoff rate in surveys is the rate at which people who start the survey breakoff, 
for whatever reason. The interview may be stopped by the respondent or by the surveyor. In the 
toplines these are indicated as refusals and hang ups. The breakoff rate is measured as the 
number of people answering the last question in the survey divided by the number of Completes. 


The opposite of the breakoff rate is the survey completion rate. 


44. The breakoff rates are extremely high in these surveys. The breakoff rates are 87.8 
percent in Arizona, 98.8 percent in Georgia, 93.5 percent in Michigan, 95.4 percent in 
Pennsylvania, and 90.6 percent in Wisconsin. In Michigan, the breakoff rate of 95.4 percent 


means that once the survey began only 4.6 percent made it to the end. 


45. The breakoff rate is a quality control indicator for survey researchers. Very high 


breakoff rates, such as observed here, are signs of quality control problems with the survey itself, 
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such as hostile or poorly trained interviewers or poorly worded questions. Any experienced 
survey researcher uses high breakoff rates to catch quality control failures. These surveys have 
extremely high rates of survey failures, which indicates that were quality control problems and 


there was no effort to correct them. 


ili. The screening question improperly allows people to take the survey who 


should not. 


46. A second substantial flaw in the survey is that the instructions allow people who are 


not affirmatively determined to be the correct person to take the survey. 


47. Past research has documented that phone surveys using registered voter lists are 
often answered by someone other than the person who was listed on the registered voter file. 
The two most common problems are that the wrong number was matched to the voter list and 
that someone other than the person the research sought to speak with answered the phone. The 


latter occurs most often with landlines.> 


48. Question | (Q1) of the survey asks “May I please speak to <lead on screen>?” 
“Lead on screen” is the name of from the voter registration list that is linked to the phone number 
that the survey has dialed. Responses to Q1 are listed as reached target, uncertain/other, refused, 


and hang up. For example, in the first table (Georgia), the responses are “Reached Target [Go to 


> Pew Research Center, “Comparing Survey Sampling Strategies: Random-Digit Dialing vs. Voter Files,” 2018. 
https://www.pewresearch.org/methods/2018/10/09/comparing-survey-sampling-strategies-random-digit-dial-vs- 
voter-files/, See page 25-26. 
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Q2]” and “[Go to Q2],” without further explanation. For other states the toplines describe this 
second response category as “Uncertain” or “What’s this about?” Importantly, both cases 


classified as “Reached Target” and as “Uncertain” are instructed to “Go to Q2.” 


49. This is an error in the branching design of the survey. People who are not 
affirmatively identified as the correct person for the interview are allowed to answer the 
remaining questions in the survey. For example, Reponses to Questions 2 and 3 show evidence 
that spouses and other family members are asked Questions 2 and 3, even though they were not 


the person whose absentee voting records are in question. 


50. A significant percent and number of respondents who are listed as not giving an 
affirmative answer to Question | are in fact kept in the survey and asked Question 2. Table 4 
shows the percent and number of respondents who were inappropriately asked Questions 2 and 3 
because they were not affirmatively identified as “the target.” This error in the survey design 
affects 13 percent of cases in Arizona and Michigan, 16 percent of cases in Pennsylvania, and 25 
percent of cases in Georgia. It is not possible to calculate the percent in Wisconsin because the 


topline report pools the “Reached Target” and “Uncertain” in a single response category. 


51. This survey branching error contaminates all of the results, and it is of sufficient 
magnitude to alter the results significantly, and perhaps explain away all of the findings entirely. 
The number of respondents in Georgia who were improperly asked Question 2 is larger than the 


number of respondents who said that they did not request an absentee ballot. In Pennsylvania it 
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explains most of the people who did not request an absentee ballot. In Arizona and Michigan, it 


can explain half of those who did not request an absentee ballot. 


52. These figures and aspects of the survey design show that the data for Q2 and Q3 were 
contaminated by improper branching from QI. This information was available to, and even 
reported by, Dr. Briggs, but he did not take them into account in calculating or interpreting his 


Error #1 and Error #2. 





Table 4. Respondents Who Were Not the Target of the Survey Were Allowed to 


Answer the Survey 




















State Percent and Number of respondents to Ql who 
were NOT the target registrant, but who were asked 
Q2 

Arizona 12.6% [N=335] 

Georgia 25.0% [N=255] 

Michigan 12.9% [N=142] 

Pennsylvania 15.7% [N=422] 

Wisconsin ** 








* The Topline Table for Wisconsin pools respondents who were coded as “Reached 
Target” and “Uncertain” and “What is this about?” It is not possible to identify how 


many Wisconsin respondents were inappropriately asked Question 2. 
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iv. Question 2 (did you request an absentee ballot) does not ascertain Permanent 


Absentee Voters or disambiguate Permanent Absentee Voters from Other Voters. 


53. Question 2 is not sufficiently clear and specific to answer the question that the 
researcher wants to answer. The survey does not ascertain whether respondents are permanent 
absentee voters or have a designated person who may request a ballot on their behalf, even 
though Arizona, Georgia, Michigan, Pennsylvania, and Wisconsin allow for some or all voters to 
be permanent absentee voters. Permanent absentee voters do not need to request a ballot in order 


for one to be sent to them for a specific election. 


55. The presence of permanent absentee voters in the registration system creates 
ambiguity in the interpretation of the question. Some permanent absentee voters may answer yes 
because the registered as for permanent absentee status, while others may say no because they do 
not need to request a ballot to receive one. The ambiguity of Question 2, and the failure to 
disambiguate permanent absentee voters from other absentee voters in the responses, introduces 
measurement error in the survey. Additional survey questions are required to distinguish 


different types of absentee voters. 


56. The measurement error will create errors in the survey that are of the form of Error #1 
described by Dr. Briggs. These are cases that would be wrongly identified as people who were 
erroneously sent a ballot, even though they did not request one. In fact, they did not need to 
request one. The survey data cannot be used to draw the conclusion that some survey 


respondents received an absentee ballot in error. 
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v. The survey cannot determine whether there was an error in handling of the 


ballot. 


57. Dr. Briggs describes a second sort of error in absentee balloting that arises because 
people say that they returned a ballot, but no absentee ballot is received or recorded by the 


election office. 


58. It is my experience working with election administrators and researching election 
administration as part of the Caltech/MIT Voting Technology Project that many absentee ballots 
are not recorded or counted because of some ballots are not received on time or are not properly 
prepared and submitted. Late absentees are not accepted, and they are usually not recorded in 
the tally of ballots received. Ballots that are spoiled, unsigned or in the incorrect envelopes or 
rejected for some other reason are not counted. The fact that there is no record of a vote or of a 
received absentee ballot is not necessarily evidence of an error in the handling of the ballot. 
Instead it may be evidence of correct treatment of ballots by the election officials in accordance 


with state laws. 


59. Question 3 does not ascertain when the ballot was mailed back or how it was mailed. 
There is no follow up question asking when the ballot was sent, whether it was signed, whether it 
was witnessed (in states where that is a requirement), and in what envelope it was sent. In short, 
the question does not allow one to determine whether or not the ballot was returned in 


compliance with state laws, and thus whether there was or was not an error in handling the 
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ballot. It is incorrect for Dr. Briggs to conclude that ballots that were not received or recorded 


are in fact errors. 


vi. Question 3 is subject to memory errors and social desirability bias. 


60. Question 3 asks people whether they voted. Specifically, it asks people who said that 
they requested an absentee ballot whether they returned an absentee ballot, that is, whether they 


voted that ballot. 


61. It has long been understood in political science that respondents to surveys over 
report voting in elections. The most commonly identified sorts of biases are memory errors and 
social desirability bias in questions asking people whether they voted.° In the context of this 


survey such biases would lead to overstatement of Yes responses to Question 3. 
C. Critique of the Survey Databases and Data Analyses 


62. There are obvious data errors and inconsistencies revealed in the Topline reports that 
are appended to Dr. Briggs’ report. As I understand his report, these are the data and reports that 
he relied on them in making his estimates and projections. Dr. Briggs states that he assumes that 
“the data is accurate.” I have examined the accounting in the Topline tables and discovered that 


the data do not add up. A routine analysis to check the consistency and integrity of data reported 


5 See for example, Allyson L. Holbrook and Jon A. Krosnick, “Social Desirability Bias in Voter Turnout Reports: Test 
Using the Item Count Technique,” Public Opinion Quarterly 74 (2010): 37-67. See also Stephen Ansolabehere and 
Eitan Hersh, ,”Validation: What Big Data Reveal About Survey Misreporting and the Real Electorate,” Political 
Analysis 20 (2012): 437-459 
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in the toplines is standard practice in the survey research field. I have performed such a check, 
and it reveals that the data lack integrity and are not correct. They should not be assumed to be 


accurate. 


i. The figures on responses to Q1 simply do not add up for the State of 


Wisconsin. 


63. The Topline table for Wisconsin reports that 2,261 people were coded as either “A- 
Reached Target” or “B-What Is This About?/Uncertain.” An additional, 1,677 respondents were 
coded as “X=Refused.” No other response categories are reported. The sum of 1,677 and 2,261 
is 3,938. The bottom of the table reports the “Sum of All Responses” is 3,495. The rows clearly 


do not total to the reported bottom line. 


64. All other survey questions and calculations for this table branch off of Question 1. 
Therefore, errors in this question infect responses to Questions 2 and 3 and make it unacceptable 
for anyone to rely on them to form conclusions. This error is a red flag for survey researchers 
indicating lack of data integrity. It should have signaled to the analyst, in this instance Dr. 


Briggs, that there is a problem with the programs that generated the data for this and other states. 


ii. The survey data for Questions | and 2 cannot be reconciled. 


65. I have examined the accounting across questions to make sure that the number of 


cases that are indicated as passing from Question | to Question 2 are the same as the number of 
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cases reported for Question 2. For Georgia, the data across questions are consistent, but for 
Arizona, Michigan, Pennsylvania, and Wisconsin there are substantial and idiosyncratic 


discrepancies. The accounting for Q1 and Q2 is shown in Table 5. 


66. First, consider Georgia. Question | has two categories: Reached Target and 
Uncertain. There are 767 Reached Target and 255 Uncertain. Those sum to 1,022. Those two 
groups are then asked Question 2. Question 2 has several response categories. There are 591 
Yes responses, 128 No responses, 175 “other” responses across various options (e.g., 
“member)[Go to Q3]’), 70 Refused, and 58 Hang ups. These sum to 1,022. For Georgia the 
total number of responses to Q2 equals the total number of respondents coded for Q2, and the 
data appear to be okay. But, looking at the other states, reveals inconsistencies that lead me to 


doubt the integrity and veracity of any of the data presented here, including Georgia. 


67. Second, consider Arizona. The topline table for Q1 has 2,147 respondents who are 
either “Reached Target” or “Uncertain” and are instructed to Go to Q2. Applying the same 
accounting used for Georgia in Arizona, there are 2,489 respondents listed in Q2. That is there 
are more than 300 respondents who answered Q2 but were not indicated in the accounting for QI 
as directed to that question. There is no other way indicated in the survey data to get to Q2 


without going through Q1. 


68. Third, consider Michigan. The topline table for Q1 has 1,100 respondents who are 


either “Reached Target” or “Uncertain.” However, there are 1,515 respondents to Q2. An 
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additional 415 people were asked Q2 than were indicated as allowed to under the branching rules 


of the survey. 


69. Fourth, consider Pennsylvania. The topline table for Q1 has 2,684 respondents who 
are either “Reached Target” or “Uncertain.” However, there are 2,537 respondents to Q2. That 


is, 147 fewer people were asked Q2 than were supposed to have been asked. 


70. Fifth, consider Wisconsin. The topline table for Q1 has 3,938 respondents who are 
either “Reached Target” or “Uncertain.” However, there are 2,723 respondents to Q2. That is, 


1,215 fewer people were asked Q2 than were supposed to have been asked. 



































Table 5. Accounting Discrepancies in the number of cases reported in Toplines for Question 1 
and Question 2 by State 
State Question | Question 2 
Number of Cases Number of Cases Difference 
“Reached Target” or “Sum of All Number (%) 
“Uncertain/Other” Responses” 
Arizona 2,147 2,489 -342* 
Georgia 1,022 1,022 0 
Michigan 1,100 1,515 -415 
Pennsylvania 2,684 2,537 +147 
Wisconsin 3,938 2,723 +1,215 
Source: Toplines appended with Dr. William Briggs’ report. 
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* Negative values mean there are fewer Reached Target or Uncertain responses to Question | 
than there are to Question 2. Positive values mean there are more Reached Target or 


Uncertain responses to Question | than there are to Question 2. 











71. I attempted to resolve this by removing refusals and hang ups, to determine whether 
this would account for apparent discrepancies. It did not. The accounting in Michigan and 
Arizona were not resolved by removing the hang ups or refusals. And, doing so created 
accounting discrepancies elsewhere. Georgia developed a deficit of cases, and the deficits in 


Pennsylvania and Wisconsin worsened. 


72. These errors in the spreadsheets will also contaminate the data in Q3, as the 
classification of respondents according to Q1 and Q2 determines whether the individual is asked 


Q3. 


73. In my experience running, designing, and analyzing large scale surveys through the 
Cooperative Congressional Election Study and serving on the board of the American National 
Election Study, errors such as these usually have two sources. This is indicative of either there 
are (1) errors in the program that the program that assigns questions to people, or (11) errors in the 
program that generates the spreadsheet. Either sort of error is catastrophic for this analysis, and 


the render the estimates, projections, and inferences in Dr. Briggs’ report entirely unreliable. 
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74. The most rudimentary check of the toplines reveals errors and inconsistencies 
throughout the data that Dr. Briggs relied on. This leads me to conclude that the data are not 


correct. Dr. Briggs erred in his assumption that they are. 


iii. There are inconsistencies in calculations. 


75. I performed a sensitivity analysis of Dr. Briggs’ calculations of the estimated ranges 
of Error #1 and Error #2. Specifically, I sought to explore how various discrepancies in the 
accounting might affect the estimates presented in Dr. Briggs’ report. The figures he presents 
are extrapolations from a few hundred survey responses to tens of thousands of absentee 
requests. Thus, errors in a few dozen cases out of the few hundred survey responses that he 


identifies as errors would be highly consequential. 


76. In performing the sensitivity analysis, I discovered that there were substantial 
inconsistencies in the way that Dr. Briggs calculated the rates of Error #1 and Error #2 using the 


survey data. 


77. Consider, first, the calculation of Error #1. I converted the first table in Dr. Briggs’ 
report from counts to percentages. I did this by dividing his lower and upper bound estimates for 
Error #1 by the total number of ballots. These are reported in the second column of Table 6. 
Second, I calculated the percent of people who responded NO or NO on behalf of their spouse to 
Question 2 and divided by the number of responses to Question 2. Third, I report two different 


Numbers of Cases used in making the calculations: the number of cases reported as “Sum of All 
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Responses” in the topline tables and that number less respondents who refused to answer. 


Finally, I calculated the percent of respondents who answered No to Q2 or whose spouse 


answered No to Q2 using the two different numbers of cases in column 4. I underline the 


number that was used by Dr. Briggs to estimate Error #1 for each state. These calculations are 


shown in the fifth column of Table 6. 









































Table 6. Calculation Inconsistencies in the Estimates for Error #1 
State Range Of Question 2 Percent of 
Error #1 Number of Cases Number of Respondents Who 
Expressed in “Sum of All Cases answered No to Q2 
Percentages Responses” 
Arizona 40.2 to 44.3% 885 No 2,489 36.4% 
21 Spouse - No 2,126 (less refusals) 42.6% 
Georgia 12.3 to 16.5% 128 No 964 14.7% 
14 Spouse - No 894 (less refusals) 15.9% 
Michigan 21.3 to 26.2% 239 No 1,515 16.9% 
17 Spouse - No 1,106 (less refusals) 23.1% 
Pennsylvania | 19.6 to 22.6% 531 No 2,537 21.9% 
25 Spouse - No 2,430 (less refusals) 22.9% 
Wisconsin 16.9 to 19.9% 379 No 222 14.1% 
4 Spouse - No 2,162 (less refusals) 17.7% 
Source: Toplines appended with Dr. William Briggs’ report. 
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78. Dr. Briggs is inconsistent in his calculations. In Georgia and Pennsylvania, the 
denominator is the sum of all responses (that is, all cases who reach Q2). But, in Arizona, 
Michigan, and Wisconsin he excludes some respondents from the total number of cases. The 
effect of excluding those cases is to inflate the estimates by 6.2 percentage points for Arizona, by 
6.2 percentage points for Michigan, and by 3.6 percentage points for Wisconsin. In Arizona and 
Wisconsin, the estimate using all cases in the denominator lies outside of the range of possible 
rates of Error #1 provided by Dr. Briggs. The estimates he offers are highly sensitive to which 
denominator he chooses to use in making his calculations. This shows a lack of rigor in 


performing the analysis that was presented. 


79. Similar inconsistencies arise in the analysis of Question 3 for the estimation of the 
rate of Error #2. Table 7 parallels Table 6, but for Question 3. The second column shows the 
ranges of Error #2 expressed in Percentages. The third column shows the Number of 
respondents who answered Yes or Yes on behalf of their spouse. The fourth column is the 
number of respondents to Q2 and to Q3. The fifth column is the Percent of Survey Respondents 


who Answered Yes to Question 3. 


80. Different denominators are used for the calculation of Error #2 in different states. In 
two instances (Georgia and Pennsylvania), Dr. Briggs uses the number of responses to Q2 as the 
denominator. In three instances (Arizona, Michigan, and Wisconsin), Dr. Briggs uses the 
number of responses to Q3, and does not adjust for refusals, as was done in Table 6. He offers 


no explanation of his calculations, or why he chose different denominators in different instances. 
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It is highly unusual to see different statistical formulas used for the computation of what is 
supposed to be the same quantity for different cases (in this instance the states) in the same 


report. The basic statistical methods deployed here lack rigor. 


81. Dr. Briggs’ estimates fail the sensitivity analysis suggested by his own calculations. 
The ranges presented in his report are not robust to variations in the formulas that he himself 
uses. In his report he reports a range of possible values for Error #1 and Error #2. Values outside 
of those ranges are highly unlikely to occur. The sensitivity analysis I have conducted reveals 
that simply using the different formulas he deploys yields values that fall outside the ranges that 
he presents. He uses the Number of Cases for Q2 in calculating Error #2 for Georgia and 
Pennsylvania and the Number of Cases for Q3 in calculating Error #2 for Arizona, Michigan, 
and Wisconsin. Consistently using the Number of Cases for Q2 produces estimated values of 
Error #2 that are below the lower bound estimates for Arizona (14.3 versus 15.2), for Michigan 
(16.0 versus 20.6), and for Wisconsin (11.9 versus 14.4). Hence, the estimated range of Error #2 
presented in Dr. Briggs’ report is not robust even to variations in the way he calculates that rate 


from the survey data. 











Table 7. Calculation Inconsistencies in the Estimates for Error #2 
State Range Of Question 2 Percent of 
Error #2 Number of Cases Number of Respondents Who 
Expressed in “Sum of All Cases answered Yes to Q3 
Percentages Responses” 
Arizona 15.2 to 18.3% 344 Yes Q2: 2,489 14.3% 
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11 Spouse - Yes Q3: 2,129 16.7% 
Georgia 22.9 to 28.2% 240 Yes Q2: 964 26.4% 

17 Spouse - Yes Q3: 623 41.3% 
Michigan 20.6 to 24.9% 232 Yes Q2: 1,515 16.0% 

10 Spouse - Yes Q3: 1,090 22.2% 
Pennsylvania | 16.3 to 19.1% 452 Yes Q2: 2,537 18.2% 

11 Spouse - Yes Q3: 1,137 40.7% 
Wisconsin 14.4 to 17.3% 316 Yes Q2: 2,723 11.9% 

9 Spouse - Yes Q3: 2,154 15.1% 
Source: Toplines appended with Dr. William Briggs’ report. 











D. Sensitivity 


82. A further exercise in sensitivity analysis is to measure the effect on the analysis of 
Q2 of the inclusion of people who should not have been included. To see the potential effect of 
the inclusion of these people in the analysis, assume that all of the people who answered 
Uncertain Q1 in fact answered No to Question 2. That is an assumption for the sake of sensitivity 


analysis. 


83. What is the potential effect of this branching error alone (excluding all other issues) 


on the survey estimates? Table 8 entertains that possibility. The Adjusted Percent who 
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Responded No to Q2 subtracts the Number of Uncertain Cases from the Numerator and 


Denominator. The rate of Error #1 cases is substantially reduced in every one of the states by the 


exclusion of these cases. In every case, the adjusted rate is far below the estimate provided in Dr. 


Briggs’ report. In Georgia, that rate falls entirely to 0. That is, the branching error can 


completely account for his Error #1 results in Georgia. 



































Table 8. Calculation Inconsistencies in the Estimates for Error #1 
State Range Of Question 2 Adjusted Percent of 
Error #1 Number of Cases Number of Respondents Who 
Expressed in “Sum of All “Uncertain” answered No to Q2 
Percentages Responses” Responses to Q1 (without 
“Uncertain” cases) 
Arizona 885 No 
40.2 to 44.3% 21 Spouse - No 335 26.7% 
Georgia 128 No 
12.3 to 16.5% 14 Spouse - No 255 0% 
Michigan 239 No 
21.3 to 26.2% 17 Spouse - No 142 13.9% 
Pennsylvania 531 No 
19.6 to 22.6% 25 Spouse - No 422 5.3% 
Wisconsin 379 No unknown No calculation 
16.9 to 19.9% 4 Spouse - No Possible 
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Source: Toplines appended with Dr. William Briggs’ report. 











E. Conclusion 


84. The estimates and projections presented by Dr. Briggs are based on survey data 
collected in Arizona, Georgia, Michigan, Pennsylvania, and Wisconsin. My overall assessment 
of these data is that they are unreliable and riddled with accounting and survey design errors. 
These errors are of sufficient magnitude and severity as to make the estimates completely 


uninformative. 


85. The data are not accurate. The Topline summaries of the survey data appended to Dr. 
Briggs’ report reveal fatal accounting errors in the data. No sound estimates or inferences can be 


drawn based on these data. 


86. The design of the survey is improperly structured so that people who were not 
identified as the Target of the study (i.e., the registered voter whose ballot is in question) were 


asked whether they had requested a ballot and whether they returned one. 


87. The magnitude of the accounting errors evident in the Toplines and the number of 


people improperly asked Questions 2 and 3 exceeds the number of people coded as Error #1 and 


Error #2. 


37 


Case 2:20-cv-13134-LVP-RSW ECF No. 36-1, PagelD.2559 Filed 12/02/20 Page 40 of 60 


88. The questions are ambiguous and do not allow researchers to determine whether 
people were permanent absentee voters or whether they submitted legally acceptable ballots in a 


timely manner. 


89. The survey respondents are highly unlikely to be representative. The surveys have 
unacceptably low response rates and breakoff rates. In Michigan specifically, 99 percent of 
people who the researchers sought to interview did not ultimately end up in the sample, 
participate in the study or complete Question 2. Scientific research standards do not allow 
researchers to assume that a sample survey a 99 percent non-response rate is representative of the 
population of interest. The breakoff rate exceeds 90 percent, indicating serious quality control 
problems in conducting the survey, which will also produce biases in the data. No effort is made 


to show that the sample is representative or to correct for possible biases. 


90. The interpretation of the surveys is deeply flawed. The interpretation of Error #1 
does not account for the large number of Permanent Absentee Voters in these states — 
approximately 2 million in Michigan. These registrants are sent a ballot without requesting one. 
The interpretation of Error #2 does not account for the fact that thousands of absentee ballots are 


not legally acceptable because they are late, spoiled, or invalid. 


91. Each of these problems would create significant biases in the estimates and 


projections offered in Dr. Briggs’ report. The multitude of weaknesses with the survey design, 


the data itself, and the interpretation lead me to conclude that no valid estimates and conclusions 
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can be made based on these data. Dr. Briggs assumed at the outset that the respondents are 
representative to the surveys and the data are accurate. Neither assumption is correct. Indeed, 
the information contained in and appended to Dr. Briggs’ report showed that to be evident. Even 
the most basic review of the information about the survey reveal the deep flaws in the design and 
errors and inconsistencies in the accounting of the survey design. These data and the analyses 


based on them do not meet the standards for scientifically acceptable research. 
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Signed at Cambridge, Massachusetts on the date below. 


Date: December 2, 2020 


Stephen Ansolabehere. 
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“Measuring the Consequences of Delegate Selection Rules in Presidential 
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The Future of Nuclear Power, Revised. 


The Future of Coal. MIT Press. Continued reliance on coal as a primary power 
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— develop a road map for technology research and development policy in order to 
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The Future of Nuclear Power. MIT Press. This cross-disciplinary study — 
drawing on faculty from Physics, Economics, Chemistry, Nuclear Engineering, 
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dioxide emissions from fossil fuel power plants. 


“Election Day Registration.” A report prepared for DEMOS. This report analyzes 
the possible effects of Proposition 52 in California based on the experiences of 6 
states with election day registration. 


Voting: What Is, What Could Be. A report of the Caltech/MIT Voting 
Technology Project. This report examines the voting system, especially 
technologies for casting and counting votes, registration systems, and polling place 
operations, in the United States. It was widely used by state and national 
governments in formulating election reforms following the 2000 election. 
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“An Assessment of the Reliability of Voting Technologies.” A report of the 
Caltech/MIT Voting Technology Project. This report provided the first 
nationwide assessment of voting equipment performance in the United States. It 
was prepared for the Governor’s Select Task Force on Election Reform in Florida. 


Chapters in Edited Volumes 
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“Taking the Study of Public Opinion Online” (with Brian Schaffner) Oxford 
Handbook of Public Opinion, R. Michael Alvarez, ed. Oxford University Press: 
New York, NY. 


“Voter Registration: The Process and Quality of Lists” The Measure of 
American Elections, Barry Burden, ed.. 


“Using Recounts to Measure the Accuracy of Vote Tabulations: Evidence from 
New Hampshire Elections, 1946-2002” in Confirming Elections, R. Michael 
Alvarez, Lonna Atkeson, and Thad Hall, eds. New York: Palgrave, Macmillan. 


“Dyadic Representation” in Oxford Handbook on Congress, Eric Schickler, ed., 
Oxford University Press. 


“Voting Technology and Election Law” in America Votes!, Benjamin Griffith, 
editor, Washington, DC: American Bar Association. 


“What Did the Direct Primary Do to Party Loyalty in Congress” (with 
Shigeo Hirano and James M. Snyder Jr.) in Process, Party and Policy 
Making: Further New Perspectives on the History of Congress, David 
Brady and Matthew D. McCubbins (eds.), Stanford University Press, 2007. 


“Election Administration and Voting Rights” in Renewal of the Voting 
Rights Act, David Epstein and Sharyn O’ Hallaran, eds. Russell Sage Foundation. 


“The Decline of Competition in Primary Elections,” (with John Mark Hansen, 
Shigeo Hirano, and James M. Snyder, Jr.) The Marketplace of Democracy, 
Michael P. McDonald and John Samples, eds. Washington, DC: Brookings. 


“Voters, Candidates and Parties” in Handbook of Political Economy, Barry 
Weingast and Donald Wittman, eds. New York: Oxford University Press. 


“Baker v. Carr in Context, 1946 — 1964” (with Samuel Isaacharoff) in 


Constitutional Cases in Context, Michael Dorf, editor. New York: Foundation 
Press. 
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2002 “Corruption and the Growth of Campaign Spending’ (with Alan Gerber and James 
Snyder). A User’s Guide to Campaign Finance, Jerry Lubenow, editor. Rowman 
and Littlefield. 

2001 “The Paradox of Minimal Effects,” in Henry Brady and Richard Johnston, eds., 
Do Campaigns Matter? University of Michigan Press. 

2001 “Campaigns as Experiments,” in Henry Brady and Richard Johnson, eds., Do 
Campaigns Matter? University of Michigan Press. 

2000 “Money and Office,” (with James Snyder) in David Brady and John Cogan, eds., 
Congressional Elections: Continuity and Change. Stanford University Press. 

1996 “The Science of Political Advertising,” (with Shanto Iyengar) in Political 
Persuasion and Attitude Change, Richard Brody, Diana Mutz, and Paul 
Sniderman, eds. Ann Arbor, MI: University of Michigan Press. 

1995 “Evolving Perspectives on the Effects of Campaign Communication,” in Philo 
Warburn, ed., Research in Political Sociology, vol. 7, JAI. 

1995 “The Effectiveness of Campaign Advertising: It’s All in the Context,” (with 
Shanto Iyengar) in Campaigns and Elections American Style, Candice Nelson and 
James A. Thurber, eds. Westview Press. 

1993 “Information and Electoral Attitudes: A Case of Judgment Under Uncertainty,” 
(with Shanto Iyengar), in Explorations in Political Psychology, Shanto Iyengar 
and William McGuire, eds. Durham: Duke University Press. 

Working Papers 

2009 “Sociotropic Voting and the Media” (with Marc Meredith and Erik Snowberg), 
American National Election Study Pilot Study Reports, John Aldrich editor. 

2007 “Public Attitudes Toward America’s Energy Options: Report of the 2007 MIT 
Energy Survey” CEEPR Working Paper 07-002 and CANES working paper. 

2006 "Constituents' Policy Perceptions and Approval of Members’ of Congress" CCES 

Working Paper 06-01 (with Phil Jones). 

2004 “Using Recounts to Measure the Accuracy of Vote Tabulations: Evidence from 
New Hampshire Elections, 1946 to 2002” (with Andrew Reeves). 

2002 “Evidence of Virtual Representation: Reapportionment in California,” (with 
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1992 


1991 


1991 


1991 


1990 


Ruimin He and James M. Snyder). 


“Why did a majority of Californians vote to lower their own power?” (with James 
Snyder and Jonathan Woon). Paper presented at the annual meeting of the 
American Political Science Association, Atlanta, GA, September, 1999. 

Paper received the award for the best paper on Representation at the 1999 Annual 
Meeting of the APSA. 


“Has Television Increased the Cost of Campaigns?” (with Alan Gerber and James 
Snyder). 


“Money, Elections, and Candidate Quality,” (with James Snyder). 


“Party Platform Choice - Single- Member District and Party-List Systems,” (with 
James Snyder). 


“Messages Forgotten” (with Shanto Iyengar). 


“Consumer Contributors and the Returns to Fundraising: A Microeconomic 
Analysis,” (with Alan Gerber), presented at the Annual Meeting of the American 
Political Science Association, September. 


“Biases in Ecological Regression,” (with R. Douglas Rivers) August, (revised 
February 1994). Presented at the Midwest Political Science Association Meetings, 
April 1994, Chicago, IL. 


“Using Aggregate Data to Correct Nonresponse and Misreporting in Surveys” 
(with R. Douglas Rivers). Presented at the annual meeting of the Political 
Methodology Group, Cambridge, Massachusetts, July. 


“The Electoral Effects of Issues and Attacks in Campaign Advertising” (with 
Shanto Iyengar). Presented at the Annual Meeting of the American Political 
Science Association, Washington, DC. 


“Television Advertising as Campaign Strategy: Some Experimental Evidence” 
(with Shanto Iyengar). Presented at the Annual Meeting of the American 
Association for Public Opinion Research, Phoenix. 


“Why Candidates Attack: Effects of Televised Advertising in the 1990 California 
Gubernatorial Campaign,” (with Shanto Iyengar). Presented at the Annual 
Meeting of the Western Political Science Association, Seattle, March. 


“Winning is Easy, But It Sure Ain’t Cheap.” Working Paper #90-4, Center for the 


American Politics and Public Policy, UCLA. Presented at the Political Science 
Departments at Rochester University and the University of Chicago. 
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1997-1998 
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2000-2001 


2001-2002 


2003-2005 


2003-2004 


2003-2005 


2006-2008 


Markle Foundation. “A Study of the Effects of Advertising in the 1990 
California Gubernatorial Campaign.” Amount: $50,000 


Markle Foundation. “An Experimental Study of the Effects of Campaign 
Advertising.” Amount: $150,000 


NSF. “An Experimental Study of the Effects of Advertising in the 1992 
California Senate Electoral.” Amount: $100,000 


MIT Provost Fund. “Money in Elections: A Study of the Effects of Money on 
Electoral Competition.” Amount: $40,000 


° 


National Science Foundation. “Campaign Finance and Political Representation.’ 
Amount: $50,000 


National Science Foundation. “Party Platforms: A Theoretical Investigation of 
Party Competition Through Platform Choice.” Amount: $40,000 


National Science Foundation. “The Legislative Connection in Congressional 
Campaign Finance. Amount: $150,000 


MIT Provost Fund. “Districting and Representation.” Amount: $20,000. 
Sloan Foundation. “Congressional Staff Seminar.” Amount: $156,000. 


Carnegie Corporation. “The Caltech/MIT Voting Technology Project.” 
Amount: $253,000. 


Carnegie Corporation. “Dissemination of Voting Technology Information.” 
Amount: $200,000. 


National Science Foundation. “State Elections Data Project.” Amount: 
$256,000. 


Carnegie Corporation. “Internet Voting.” Amount: $279,000. 


Knight Foundation. “Accessibility and Security of Voting Systems.” Amount: 
$450,000. 


National Science Foundation, “Primary Election Data Project,” $186,000 
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2008-2009  Pew/JEHT. “Measuring Voting Problems in Primary Elections, A National 
Survey.” Amount: $300,000 

2008-2009 =Pew/JEHT. “Comprehensive Assessment of the Quality of Voter Registration 
Lists in the United States: A pilot study proposal” (with Alan Gerber). 
Amount: $100,000. 

2010-2011 National Science Foundation, “Cooperative Congressional Election Study,” 
$360,000 

2010-2012 Sloan Foundation, “Precinct-Level U. S. Election Data,’ $240,000. 

2012-2014 National Science Foundation, “Cooperative Congressional Election Study, 2010- 
2012 Panel Study” $425,000 

2012-2014 National Science Foundation, “2012 Cooperative Congressional Election 
Study,” $475,000 

2014-2016 National Science Foundation, “Cooperative Congressional Election Study, 2010- 
2014 Panel Study” $510,000 

2014-2016 National Science Foundation, “2014 Cooperative Congressional Election 
Study,” $400,000 

2016-2018 National Science Foundation, “2016 Cooperative Congressional Election 
Study,” $485,000 

2018-2020 National Science Foundation, “2018 Cooperative Congressional Election 
Study,” $844,784. 

2019-2022 National Science Foundation, RIDIR: “Collaborative Research: Analytic Tool 
for Poststratification and small-area estimation for survey data.” $942,607 

Professional Boards 


Editor, Cambridge University Press Book Series, Political Economy of Institutions and 
Decisions, 2006-2016 


Member, Board of the Reuters International School of Journalism, Oxford University, 2007 to 


present. 


Member, Academic Advisory Board, Electoral Integrity Project, 2012 to present. 
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Contributing Editor, Boston Review, The State of the Nation. 
Member, Board of Overseers, American National Election Studies, 1999 - 2013. 
Associate Editor, Public Opinion Quarterly, 2012 to 2013. 


Editorial Board of Harvard Data Science Review, 2018 to present. 

Editorial Board of American Journal of Political Science, 2005 to 2009. 

Editorial Board of Legislative Studies Quarterly, 2005 to 2010. 

Editorial Board of Public Opinion Quarterly, 2006 to present. 

Editorial Board of the Election Law Journal, 2002 to present. 

Editorial Board of the Harvard International Journal of Press/Politics, 1996 to 2008. 
Editorial Board of Business and Politics, 2002 to 2008. 

Scientific Advisory Board, Polimetrix, 2004 to 2006. 


Special Projects and Task Forces 

Principal Investigator, Cooperative Congressional Election Study, 2005 — present. 
CBS News Election Decision Desk, 2006-present 

Co-Director, Caltech/MIT Voting Technology Project, 2000-2004. 

Co-Organizer, MIT Seminar for Senior Congressional and Executive Staff, 1996-2007. 
MIT Energy Innovation Study, 2009-2010. 

MIT Energy Initiative, Steering Council, 2007-2008 

MIT Coal Study, 2004-2006. 

MIT Energy Research Council, 2005-2006. 


MIT Nuclear Study, 2002-2004. 
Harvard University Center on the Environment, Council, 2009-present 


Expert Witness, Consultation, and Testimony 


2001 Testimony on Election Administration, U. S. Senate Committee on Commerce. 

2001 Testimony on Voting Equipment, U.S. House Committee on Science, Space, 
and Technology 

2001 Testimony on Voting Equipment, U.S. House Committee on House 
Administration 

2001 Testimony on Voting Equipment, Congressional Black Caucus 

2002-2003. McConnell v. FEC, 540 U.S. 93 (2003), consultant to the Brennan Center. 

2009 Amicus curiae brief with Professors Nathaniel Persily and Charles Stewart on 


behalf of neither party to the U.S. Supreme Court in the case of Northwest 
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2009 
2011-2015 


2011-2013 


2012-2013 


2011-2012 


2012 


2012-2014 


2013-2014 


2013-2014 


2013-2015 


2014 


2014- 2016 


2015 


2016-2017 


2017-2018 
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Austin Municipal Utility District Number One v. Holder, 557 U.S. 193 (2009). 
Testimony on Voter Registration, U. S. Senate Committee on Rules. 

Perez v. Perry, U.S. District Court in the Western District of Texas (No. 5:11- 
cv-00360). Exert witness on behalf of Rodriguez intervenors. 

State of Texas v. United States, the U.S. District Court in the District of 
Columbia (No. 1:11-cv-01303), expert witness on behalf of the Gonzales 
intervenors. 

State of Texas v. Holder, U.S. District Court in the District of Columbia (No. 
1:12-cv-00128), expert witness on behalf of the United States. 

Guy v. Miller in U.S. District Court for Nevada (No. 11-OC-00042-1B), expert 
witness on behalf of the Guy plaintiffs. 

In re Senate Joint Resolution of Legislative Apportionment, Florida Supreme 
Court (Nos. 2012-CA-412, 2012-CA-490), consultant for the Florida 
Democratic Party. 

Romo v. Detzner, Circuit Court of the Second Judicial Circuit in Florida (No. 
2012 CA 412), expert witness on behalf of Romo plaintiffs. 

LULAC v. Edwards Aquifer Authority, U.S. District Court for the Western 
District of Texas, San Antonio Division (No. 5:12cv620-OLG,), consultant and 
expert witness on behalf of the City of San Antonio and San Antonio Water 
District 

Veasey v. Perry, U. S. District Court for the Southern District of Texas, Corpus 
Christi Division (No. 2:13-cv-00193), consultant and expert witness on behalf of 
the United States Department of Justice. 

Harris v. McCrory, U. S. District Court for the Middle District of North 
Carolina (No. 1:2013cv00949), consultant and expert witness on behalf of the 
Harris plaintiffs. (later named Cooper v. Harris) 

Amicus curiae brief, on behalf of neither party, Supreme Court of the United 
States, Alabama Democratic Conference v. State of Alabama. 

Bethune-Hill v. Virginia State Board of Elections, U. S. District Court for the 
Eastern District of Virginia (No. 3:2014cv00852), consultant and expert on 
behalf of the Bethune-Hill plaintiffs. 

Amicus curiae brief in support of Appellees, Supreme Court of the United 
States, Evenwell v. Abbott 

Perez v. Abbott, U.S. District Court in the Western District of Texas (No. 5:11- 
cv-00360). Exert witness on behalf of Rodriguez intervenors. 

Fish v. Kobach, U. S. District Court in the District of Kansas (No. 2:16-cv- 
02105-JAR). Expert witness of behalf of the Fish plaintiffs. 
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